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ABSTRACT 

The cost effectiveness of using certified nail as a 
device to raise response rates in nail surveys is investigated. Of 
the three optional services offered by the Post Office when sending 
certified aail, the base certified rate vas reconoended, since the 
Post Office often failed to p^rfors the aore expensive services. A 
saople of 120 students from 3p schools vas used, vith two males and 
two fenales participating fro^ each school, continuing for 19 days 
after the nail-out date, the 4xperinent randoaly assigned people fcon 
the sane sex and school to either a certified nail group or a 
first-class nail giroup. The ovjerall difference in response rates 
appeared to be appreciable. Tie cost effectiveness of using certified 
nail for a 10 iten and a 150-itea questionnaire was then fornulated. 
The results indicated that th4 nost cost effective plan was the use 
of certified nail with the lo fig questionnaire and first-class nail 
with the short one. It was reconnended that the cost effectiveness, of 
nail survey plans be calculated prior to their inplenentation. 
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The Use of Certified Mail in Mail Surveys 

One frequently used technique that attempts to raise response rates 
in mail surveys is sending survey material by certified mail. Astin and 
Panos (1969) have reported the effectiveness of certified mail as a device 
to raise response rates to a level significantly higher than those obtained 
with first class mail. 

Certified mail was used extensively in mailing survey materials 
to participants in the National Longitudinal Study of the Class of 1972 (NLS), 
conducted by ETS for the U. S. Office of Education. Certified mail was 
used because it was believed that initial response rates with certified 
mail would be high and thus the cost of follow-up low. A survey plan with 
two waves of questionnaires then a telephone follow-up to all nonrespondents 
was used to collect data. Identical plans were used for each mail survey 
performed in NLS. 

Although using certified mail typically has been found to result in 
higher response rates, it is also a costly and time-consuming procedure. 
Postage for mailing must be prepared clerically, rather than by automated 
procedures, and the amount of preparation and record-keeping is doubled. 
Thus, an experiment was conducted, designed to assess the cost-effectiveness 
of cer'.ified mail. This paper documents the results of that experiment. 
Certified Mail 

Before proceeding, the various possible options afforded by certified 
mail should be pointed out, as well as some operational considerations. 
The rost of certified mall Is $0.30 plus postage. When this form of 
certified mail is used, mail is delivered to the address given, anyone at 



that address signs a card indicating receipt of the mail, and that card 
is then returned to the sender. Two pieces of information must be placed 
on an envelope prior to mailing. One piece is a certified number and 
corresponding receipt, that has space for recording the addressee and must 
be retal d for recording purposes. The second piece is the card that is 
returned when the piece of mail is delivered. 

Two optional services are offered with certified mail for additional 
fees. Both relate to additional return receipt service. One option results 
in the returned card showing to whom and the date the piece of mail was 
delivered ($0.15), and the other option results in the card showing to 
whom, date, and where the piece was delivered ($0.35). Each of the two 
options can expanded to include delivery to the addressee only for an 
additional $0.50. 

In this study, certified mail with the option of showing to whom, 
date, and where delivered was used. Thus, the total mailing coat was 
$0.65 ($0.30 certified rate + $0.35 second option) plus postage of $0.22 
in this case* In future cases where researchers desire to use certified 
mail, the authors recommend using only the base certified rate. The 
authors frequently noted that services requested were not performed by tlie 
Post Office* 
Design 

The experiment described in this report was part of a survey of student 
from 152 high schools, with four students sampled from each school. A 
subsample of 30 schools was selected to be in the experiment, giving a 
student sample size for* this experiment of 120 people. Within each school 
sample, there were two males and two females. People of the same sex and 



school were randomly assigned to either a certified mail s^o^^P or-a f^gular 
mail group. The experiment was continued for 19 days after the mail-out 
date. 
Results 

The responses for each design category are given in Table 1. There 
are 30 subjects in each cell of the table, /flie entries in the cells 



represent the actual number of questionnaires returned for that particulat-- 
combination. A response rate of 41,7^ (25 out of 60) was achieved for 
the certified mail group, and a response rate of 26.7% (16 out of 60) was 
obtained for the first class mail group. Although chi-square test'? 
indicated that there were no significant differences (p<.05) in response 
between certified mail and first class mail, males and females, and the 
interaction betx^een the two, the overall difference in response rates 
appears to be appreciable. The failure to reach significance is most likely 
related to the small number of people in the experiment, and we will assume 
response rates approximately equal to the estimates above will be equalled 
in practice. 



Table I ^ 



Responses for Each Design Category 



Sex 



First Class Mail 



Certified Mail 



Total 



Male 



6 



12 



18 



Female 



10 



13 



23 



Total 



16 



25 



41 



Cos t Ef f ec t Ivenes^^^ 

Suppose It Is assumed that the response rates obtained above hold 
for both mailing waves in surveys of the form described above* Although 
this assumption is somewhat unrealistic, especially if results from the 
first wave show a high response rate, it will serve for an illustration» 
Other assumed response rates may be substituted in the computations that 
follow. The question then becomes: Is the use of certified mail to 
increase response rates worth the additional cost? 

Assuming a survey model with two mailings and a complete telephone 
follow-up, the authors have estimated the general administrative cost 
(including overhead) of preparing materials for mailing, keeping records, 
and checking-in data, to be $2.00 per student for first class mail and 
$3.50 per student for certified mail. The total cost for a survey may 
be estimated by the formula 

Total Cost « Initial Cost + 1st Mail Cost + 2nd Mail Cost 
Telephone Follow-up Cost. 

We will illustrate use of determining the total cost with two cases. 
One case involves a short questionnaire with less than ten items and assumes 
telephone cost to be $10.00 per follow-up student. The other case will 
involve a long questionnaire with approximately 150 individual items, 
which assumes a cost of $15.00 per follow-up student. Of course, other 
cases could be designed, and the merits of certified mail versus first class 
mall should be assessed for individual surveys. 
Case It Short puestionnaire 

Four possibilities exist for using first class mall and certified 
mail. As: mlng a 41.7% response rate for certified mail and a 26.7% 
response rate for first class mall and $0.16 postage, the total costs for 
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the four combinations of mailing are approximately: 

(a) 3.5N + .81N + (.583)(.81)N + [1-(.417 + (.417) C. 583)) ) x ION 

s $8.182N (Certified mail on both mailings); 
Initial Cost (3.5N) + 1st Mail Cost (.81N) + 2nd Mail Cost ((.583) (.81)N) + 
Telephone Follow-up Cost ([1-(.417 + (.417) (.583)) ] x ION). Initial 
cost is determined by taking the general administrative cost multiplied 
■ by the number in the survey. Each person in the survey will receive 

a piece of mail on the first wave with a postage of $0.81 = $0.65 + $0.16. 

^ Since 41.7% will return surveys, 1 - .417 = .583 represents the 

proportion of the total sample who will receive a second questionnaire. 
Since 41.7% of those sent materials on the first will respond and 
41.7% of the .583N who were sent materials on a follow-up will respond, 
the number of individuals requiring phone calls will be [lr(.417 + (.417) 
(.583)) ]N. 

(b) 2. ON + .16N + (.16)(.733)N + [l-(.267 + (.267)(.733))3 x ION 
= $7.647N (First" class mail on both mailings); 

(c) 2. ON + .16N + (3.5-2.0)(.733)N + (.81)(.733)N + (l-(.267 + 
(.417) (.733))) X ION - $8.124N (Certified mail on second mail 
only) ; and 

(d) (3.5-(3.5-2.0)(.583))N + (.81)N + (.16)(.583)N + ll-(.417 + 
(.267)(.583))] X ION = $7.802N (Certified mail on first mail only). 

Using these cost and response rate assumptions, it appears as though the 
most cost-effective plan is to use first class mall on both mailings, with 
a complete telephone follow-up. 
Case Hi Long Questionnaire ^ 

The total costs for a survey of N subjects, assuming mailing rates 

of $0.40, are: 



(a) 3.5N+ 1.05N + (.583) (1.05)N + [1-(.417 + (.417) (.583)) ] x 15N 
= $10.262N (Certified mail on both mailings); 

(b) 2. ON + .4N + .4(.733)N + [l-(.267 + (.267) (.733)) ] x 15N 
= $10,748 (First class mail on both mailings); 

(c) LON + .40N + (3.5-2.0)(.733)N + (1.05) (.733)N + [l-(.267 + 
/ 417) (.733))] X 15N = $10,675 (Certified mail on second 

mailing only) ; and \ 

j 

(d) (3.5-(3.5-2.0)(.583))N + (1.05N + (.4)(.583)N + [1-(,417 + 
(•267)(.583))] X 15N = $10,313 (Certified mail on first mail 
only) . 

On the basis of these calculations, it does appear that certified 
mail on both mailings is a cost-effective procedure. 
Conc lus^ions 

One could make other calculations with other assumptions and cost 
figures to determine the cost-effectiveness of a given plan. As can 
be seen from the above cases, certified mail is not necessarily more 
cost-effective than first class mail. In general, when telephone follow-ups 
are expensive, certified mail enjoys a slight advantage. If only the 
base cost for certified mail were used ($0.30 plus postage), certified mail 
might fare better. The authors recommend that the cost-effectiveness of 
alternative mail survey plans be considered, in the manner illustrated 
above, prior to their either being proposed or implemented. 
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